Shared annotation system and method

ABSTRACT

The invention relates generally shared annotation systems. More particularly, the invention provides a method for automatically navigating a document in a display having at least a first portion and a second portion, the method comprising: receiving an annotation related to the document, the annotation generated by a user at a first client; associating the annotation with a first indication in the document; receiving, from a user at a second client, an input to navigate a first portion of a display at the second client, the input causing the first indication to be displayed in the first portion of the display; and in response to the input, automatically displaying the annotation in a second portion of the display at the second client.

COPYRIGHT NOTICE

A portion of the disclosure of this patent document contains materialwhich is subject to copyright protection. The copyright owner has noobjection to the facsimile reproduction by anyone of the patent documentor the patent disclosures, as it appears in the Patent and TrademarkOffice patent files or records, but otherwise reserves all copyrightrights whatsoever.

BACKGROUND OF THE INVENTION

The inventions disclosed herein relate generally to collaborativesystems and more particularly to shared annotation systems.

Users often wish to collaborate on shared documents in a network. Forexample, in a business environment, users at different companies maycollaborate on a business agreement such as creating a contract or alicense agreement.

One issue associated with network collaboration is synchronicity. Forexample, users often collaborate by exchanging versions of documents viae-mail or other similar means. A first user edits or otherwise commentsa document and then sends the revised version to a second user forfurther input. The second user makes or otherwise provides their inputand then e-mails the new document back to the first user. While thefirst user is editing the document, however, the second user cannotprovide input since they do not possess the current version of thedocument (currently being edited by the first user) and therefore do notknow what changes the first user might be making. Similarly, the firstuser cannot provide further input while the document is being edited bythe second user. It is thus desirable for users to be able to providesynchronous comments and edits without having to wait for other users.

Another issue associated with network collaboration is applicationheterogeneity. In existing systems, users must have the same specializedcollaboration software in order to collaborate and share information.For example, one current collaborative system by iMarkup Solutions ofVista, Calif. requires both users to download and install a specializedplug-in in order to extend collaborative functionality to the usersystems. Many users find this technically challenging to configure orsimply inconvenient. It is thus desirable for users to be able tocollaborate using tools that are application agnostic and do not requireadditional specialized software.

U.S. Pat. No. 6,438,564 discusses a system which allows users toassociate discussions within documents. Discussions include comments,annotations, and notes and are associated with documents by associatingthe discussion with a document identifier. Discussions are storedseparately from their related documents. When a particular document isrequested by a user, any related discussions associated with theidentifier for the document are also retrieved. The system discussed inthe '564 application has a number of shortcomings. For example, in the'564 patent, only HTML text associated with a discussion is stored. Ifthe discussion is linked to another item, for example a media item, suchas a graphic, a video clip, an audio clip, etc., the media file is notstored in the system database containing the HTML text and other dataassociated with the discussion. Also, only a link to the media isstored. Thus, if a user desires to use a media item in a discussion,they must first upload the item to a separate web server or else thelink in the '564 patent system database to the item will be invalid.This presents users with a significant inconvenience. Further, thesystem only parses HTML tag data such as paragraphs, lists, images, andtables, to determine a location for a discussion within a document.Discussions are thus limited to hanging off of paragraphs, lists,images, tables, etc. and a user is not, for example, able to link adiscussion to an arbitrary word or phrase within the document. This lackof flexibility limits the user's ability to freely comment within adocument and also presents a significant limitation with respect to thelevel of granularity at which a given document may be discussed. Usingthe '564 patent system, for example, a user could not comment onindividual words in a poem which might be highly desirable given theimportance of individual word choice in poetry.

There is thus a need for systems and methods which are applicationagnostic and allow users to synchronously share annotations regarding aparticular document. There is also a need for systems and methods whichpermit users to place annotations at any arbitrary location within adocument.

SUMMARY OF THE INVENTION

The present invention addresses, among other things, the problemsdiscussed above with shared annotation systems. In accordance with someaspects of the present invention, computerized methods are provided forenabling a plurality of users to collaborate or otherwise provideannotations and other input and feedback related to shared documents andcontent in a computer network. Users are able to synchronously navigatecontent via multi-portion displays in which indicators related to theannotations are embedded in document content in a first portion of thedisplay and the related annotations are synchronously presented in atleast a second portion of the display. In some embodiments, the systemalso generates custom documents based on annotated content, providescommerce opportunities related to annotated content, persistentlypresents selected multimedia content while navigating a plurality ofdocument pages, and accepts and indexes annotations related to visualcontent elements such as graphics and photographs.

In one embodiment, the system enables a method for automaticallynavigating a document in a display having at least a first portion and asecond portion, the method comprising: receiving an annotation relatedto the document, the annotation generated by a user at a first client;associating the annotation with a first indication in the document;receiving, from a user at a second client, an input to navigate a firstportion of a display at the second client, the input causing the firstindication to be displayed in the first portion of the display; and inresponse to the input, automatically displaying the annotation in asecond portion of the display at the second client.

In some embodiments, the display comprises a browser window, such as anInternet browser. In some embodiments, the document comprises anelectronic book, a digital photo album containing one or more digitalphotos, a web page, a text document, or a multimedia document. In someembodiments, the annotation comprises a text annotation, such as acomment related to the document. In other embodiments, the annotationcomprises a graphical annotation, such as a photograph. In otherembodiments, the annotation comprises an audio annotation, a videoannotation, a multimedia annotation, or a discussion group related tothe document. In some embodiments, the input comprises an input toscroll the first portion of the display or an input to navigate to aportion of the document containing the first indication. In someembodiments, the first indication comprises a graphical indication, suchas an icon. In some embodiments, receiving an annotation comprisesreceiving form data submitted by the user at the first client, such asreceiving HTML form data.

In some embodiments, associating the annotation with a first indicationin the document comprises: identifying a portion of the document towhich the annotation relates; and associating the first indication withthe portion of the document to which the annotation relates. Forexample, in some embodiments, the annotation comprises a discussiongroup related to the portion of the document. In some embodiments, theannotation is added to a data structure stored in memory, the datastructure comprising a list of annotations relating to portions of oneor more documents. In some embodiments, the list of annotationscomprises a list of bookmarks. In some embodiments, the system receivesinput selecting an annotation from the list of bookmarks and displays,in the first portion of the display, at least a portion of a document towhich the annotation is related and displays at least the selectedannotation in the second portion of the display.

In some embodiments, associating the first indication comprisesembedding the first indication in the portion of the document to whichthe annotation relates. In some embodiments, embedding the firstindication comprises: receiving location data related to the portion ofthe document; processing the location data to determine a first locationwithin the document relative to a location of the portion within thedocument; and generating a new version of the document, the new versionof the document containing the first indication embedded at the firstlocation. For example, in some embodiments, the location data comprisesone or more from the group comprising: a document identifier, a sectionidentifier, a chapter identifier, a bookmark identifier, a portionlength, and a portion offset.

In some embodiments, the invention also includes systems and methods forreplacing a first version of the document stored in memory with the newversion of the document, for example by overwriting a first version ofthe document with a new version of the document.

In some embodiments, receiving an annotation comprises receiving anannotation related to an image contained in the document, for examplereceiving information identifying one or more subjects of the image. Insome embodiments, the system also includes methods for associating theone or more subjects with the image, such as by updating a datastructure stored in memory, the data structure storing associationsbetween one or more images and one or more subjects of the one or moreimages.

In some embodiments, the annotation comprises a commercial offer, suchas an offer to purchase a product related to the document. In someembodiments, the system also includes methods for processing a requestby a user at a client to purchase the product, such as methods fortransmitting the product and the document to the user. In someembodiments, the system also includes methods for communicating, to auser at a client, an offer to purchase the document and a set ofannotations related to the document, such as a set of annotationsselected by the user. The system processes the user request to purchasethe document and the set of annotations, for example by printing thedocument and the set of annotations. In some embodiments, for eachannotation related to a portion of the document, the system prints theannotation and the related portion of the document on the same page. Insome embodiments, processing the user request comprises transmitting thedocument and the set of annotations to the user.

In some embodiments, the system also includes methods for authenticatingthe user at a first client and authorizing the user at the first clientto provide the annotation; and authenticating the user at the secondclient and authorizing the user at the second client to navigate thedocument.

In accordance with another aspect of the present inventions, the systemincludes methods to annotate content of a web page. An indication isinserted in and associated with content according to markup languagedescribing offsets including a starting point and an endpoint for theindication, the starting point and endpoint offsets corresponding to anumber of characters from a location within the content. In someembodiments, the system includes program code that captures user inputsidentifying selections according to a paragraph identifier, a startingpoint value, and an ending point value. In some embodiments, the systemenables a method for selecting an arbitrary string of characters on aweb page and posting the selection, including related metadata, to anapplication server. In some embodiments, the related metadata includespositional metadata and content identifiers.

In one embodiment, the system enables a method for creating a custommemory book including original content supplied by a first party,annotations provided by one or more users, and multimedia elementsprovided by other users. For example, in some embodiments, users createa memory book by customizing existing content provided by contentcreators. In some embodiments, the original article also generallycontains indications and corresponding annotations input by varioususers responding to the original article. A user can then create anynumber of custom memory books from the original article by uploadingadditional multimedia elements and selecting specific annotations toinclude in their personal memory book. In some embodiments, a useruploads their own personal pictures to replace or supplement thepictures in the original article posted by the content provider. In someembodiments, a user also uses pictures posted as annotations by otherusers to replace or supplement pictures of the original article or theyuse additional pictures provided by the content provider or othercontent providers. In some embodiments, users also select customannotations to include with the memory book by filtering or otherwiseselecting annotations from the set of annotations posted by other usersregarding the original article. In one embodiment, a user automaticallyselects annotation from a list of friends who post annotations. In otherembodiments, users select annotations individually or based on criteriasuch as ratings from other users or annotation type. In someembodiments, the system enables a method for printing and binding thecustom memory book, such as by using standard book publishing equipmentand techniques.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention is illustrated in the figures of the accompanying drawingswhich are meant to be exemplary and not limiting, in which likereferences are intended to refer to like or corresponding parts, and inwhich:

FIG. 1 is a block diagram of a shared annotation system according to anembodiment of the present invention;

FIG. 2 is a block diagram of functional modules in a shared annotationsystem according to an embodiment of the present invention;

FIG. 3 is a flow chart of a method to synchronously navigate sharedannotations according to an embodiment of the present invention;

FIG. 4 a is a block diagram of an exemplary screen display of a sharedannotation system according to an embodiment of the present invention;

FIG. 4 b is a block diagram of two exemplary screen displays of a sharedannotation system according to an embodiment of the present invention;

FIG. 5 is a flow chart of a method for processing an annotationaccording to an embodiment of the present invention;

FIG. 5A presents an exemplary sample of code for an XHTML formatted pageof content according to one embodiment of the invention;

FIG. 5B presents an exemplary sample of code for an XHTML formatted pageof content according to one embodiment of the invention;

FIG. 6 is a flow chart of a method of annotating a visual elementaccording to an embodiment of the present invention;

FIG. 6A is a flow chart of a method of recreating a page of contentaccording an embodiment of the invention;

FIG. 6B is a flow chart of a method of processing an element during pagecreation according to an embodiment of the invention;

FIG. 7 is a flow chart of a method of providing a customized documentrelated to a shared annotation system according to an embodiment of thepresent invention;

FIG. 8 is a block diagram of a sample page from a customized documentrelated to a shared annotation system according to an embodiment of thepresent invention;

FIG. 8A is a screenshot of an exemplary article page of a memory bookaccording to an embodiment of the present invention;

FIG. 8B is a screenshot of an exemplary comments page of a memory bookaccording to an embodiment of the present invention;

FIG. 8C is a screenshot of an exemplary dynamic print page according toan embodiment of the invention; and

FIG. 9 is a flow chart of a method of presenting a selected multimediaelement while navigating a document in shared annotation systemaccording to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Preferred embodiments of the invention are now described with referenceto the drawings. As described further below, systems and methods arepresented regarding a shared annotation system. A plurality of userscollaborate or otherwise provide annotations and other input andfeedback related to shared documents and content in a computer network.Users are able to synchronously navigate content via multi-portiondisplays in which indicators related to the annotations are embedded indocument content in a first portion of the display and the relatedannotations are synchronously presented in at least a second portion ofthe display. In some embodiments, the system also generates customdocuments based on annotated content, provides commerce opportunitiesrelated to annotated content, persistently presents selected multimediacontent while navigating a plurality of document pages, and accepts andindexes annotations related to visual content elements such as graphicsand photographs. Additional aspects and features of the system will alsobe appreciated by one skilled in the art as further described below.

FIG. 1 presents a block diagram of a shared annotation system accordingto an embodiment of the present invention. As shown, the system includesone or more clients including first client 105, a second client 110, andan nth client 115, connected to a network 120, a content server 125including a content processor 130 communicatively coupled to a datastore 135, and one or more additional computers including a moderatorcomputer 140, an administrator computer 145, and a support computer 150.Clients 105, 110, and 115, and other computers in the system, includingpersonal computers and other computing devices known in the artincluding personal digital assistants (“PDAs”), tablet computers,cellular telephones, and other devices. The clients are communicativelycoupled to the content server 125 via a computer network 120, such asthe Internet or a local area network (“LAN”). Users of the client'sdevices collaborate or otherwise provide annotations and other input andfeedback related to shared documents and content in the network. Theusers collaborate or otherwise provide annotations regarding the contentvia one or more software modules including a display module. Forexample, in some embodiments users interact with content and provideannotations via a web browser, such as Microsoft Internet Explorer orNetscape Navigator.

The content server 125 contains a content processor 130 and othermodules directed to receiving and processing user requests regardingcontent. Requests include annotations regarding content, requests fornew content, navigation inputs regarding content, and other userrequests. The content server 125 is communicatively coupled to a datastore 135. The data store 135 stores a variety of data includingdocument content for delivery to users, user account and registrationinformation, annotations and other information generated by usersregarding content, and other related data. As used herein, annotationsgenerally include content-related input provided by users including textinput, graphical input, audio input, video input, and other types ofinput, associated in some way with a particular selected charactersequence in a primary set of content. For example, a user may input atextual comment or a user may upload a picture related to content. Auser may also provide a voice recording or other recording related tocontent or even a video clip as an annotation. Annotations may alsoinclude a discussion group or other similar forum or means to facilitatethreaded discourse or other interaction between users regarding aparticular portion of a document. For example, a user may find aparticular paragraph of a document very important and create alocation-specific discussion group regarding the paragraph as anannotation.

Additional computers are also connected to the network 120 and interfacewith content server 125 and client computers to provide additionalfunctionality. For example, moderator computer 140 may be used by amoderator to review and approve user comments and annotations. Anadministrator computer 145 may manage other aspects of user interactionwith the system such as user registration or security related issues.Support personnel may use support computer 150 to interface with usersand provide additional assistance or help regarding user concerns.Additional computers of remote clients may also be employed or used byrole-based personnel such as a picture moderator, a comments moderator,a topic approver, a new edition creator, a discussion group moderator,etc.

FIG. 2 presents a block diagram of functional modules in a sharedannotation system according to an embodiment of the invention. Thesystem is implemented using Model View Controller (“MVC”) architectureas known in the art. Four tiers are presented including a client tier153, a presentation tier 163, and an application tier 167, as well as adata store 135 or integration tier containing the data model. In someembodiments, modules are distributed among one or more content servers125 and clients 105, 110, 115. The system may also implement multipletiers and distribute modules to distribute functionality in order toimprove system efficiency or otherwise load balance processingoperations.

The client tier 153 includes a highlight module 155, a synchronizationmodule 157, an annotation module 159, and a view modes module 161. Theclient tier includes code, such as JavaScript code, that executes onvarious pages, such as DHTML pages. The highlight module 155 isgenerally directed to managing selection and highlighting of annotationsand text in the original content. For example, if a user clicks on animage annotation, the highlight module manages highlighting thecorresponding text in the first portion of the display as well as theimage annotation in the second portion of the display. Conversely, if auser selects or otherwise interacts with an annotation in the secondportion of the display, the corresponding text or other visual elementsare highlighted in the first portion of the display by the highlightmodule.

The synchronization module 157 manages relationships between originalcontent in the first portion of the display and correspondingannotations in the second portion of the display. In the second portionof the display, annotations are presented corresponding to content inthe first portion of the display as the user scrolls the first portionof the display. Similarly, when the user scrolls the second portion ofthe display containing annotations, the first portion of the displayalso synchronously scrolls ensuring that original content in the firstportion corresponding to the annotations in the second portion isconsistently displayed. The synchronization module 157 also preventsunnecessary scrolling which might cause flicker. For example, noscrolling is performed if an icon or other indication present in thefirst portion of the display corresponds to an annotation alreadyvisible in the second portion of the display. Thus, the second part ofthe display is scrolled to find the next annotation only when anavigation input changes the display such that an indication in thefirst portion of the display disappears and vice-versa.

The annotation module 159 generally manages and processes annotations ofimages and other multimedia content. For example, when a user selects aphoto for annotation, the annotation module 159 presents a rectangularselection box over the photo that may be resized to precisely indicatethe portion of the photo to which an annotation refers. Multipleselection rectangles or other selection shapes may be drawn over a photoeach corresponding to individual annotations. Upon receipt of anappropriate input, for example when a save or post annotation(s) buttonis selected, the annotation module also handles communicating theselection input(s) and related annotation information to other modulesof the system as further described herein.

The view modes module 161 generally manages and controls presentationmodes for content. For example, the view modes module switches betweenmodes such as “embedded mode” in which indications or icons arepresented inline with original content, “non-embedded mode” in whichindications are presented to the left of the original content with oneindication type per paragraph, and “memory book mode” in whichindications are aggregated by type and presented inline at the end ofindividual paragraphs as opposed to directly in the text or to the leftof the text.

The presentation tier 163 generally includes a number of modules 165running code within the web container. The code modules 165 generallyinclude a controller responsive to data and inputs received from theclient tier 163 as well as the business tier 167. Exemplary code modules165 correspond to modules of the business tier 167 as further describedherein and include a back office module, a book module, a bookmarkmodule, a comment module, a conversion module, an ecommerce module, aprint module, an image module, a media module, a profile module, asearch module, and a user management module. Code modules 165 provide abridge between application logic provided by the business tier andclient inputs or presentation outputs.

The business tier 167 generally includes a number modules including aback office module 169, a book or content module 171, a bookmarks module173, a user management module 175, a content serialization module(“CSE”) 177, a media module 179, a comments module 181, a statisticsmodule 183, a conversion module 185, an ecommerce module 187, a printmodule 189, a personalization module 191, a profile module 193, a searchmodule 195, a payment gateway module 197, and a print services module199. These various modules support a variety of internal administrativeoperations and actions, as well as process and respond to user actionsin the presentation tier.

The user management module 175 is generally responsible for handlinguser-related operations such as registration, authentication, andmembership rights and approvals (such as for administrators, regularmembers, etc.).

The book or content module 171 generally manages and directscontent-related operations such as navigation to other pages andtracking user preferences. For example, the content module 171 trackspreferred viewing modes and last pages visited for users. Generally, thecontent module is not directly responsible for serving content, however,since this is handled and resolved in the presentation tier by thecorresponding book code module of the presentation tier and other codemodules for the sake of improved performance.

The bookmarks module 173 generally manages the user's private bookmarkslist for content and annotations. For example, the bookmarks module 173maintains a data structure containing pointers to locations for contentor annotations that a user may wish to revisit or otherwise mark as afavorite. When an input is received selecting a bookmark, the systemautomatically navigates to and presents the related content orannotation corresponding to the selected bookmark.

The comments module 181 is generally directed to processing operationsassociated with posing annotations. For example, the comments module 181manages inputs posting or replying to annotations, applying automaticmoderation to posted annotations, and notifying moderators whenannotations trigger various notification filters. In some embodiments,the comments module 181 also notifies annotation authors when a reply orother corresponding annotation is posted regarding their authoredannotation. Similarly, media module 179 processes graphical annotationsand other graphical information provided by users. For example, themedia module 179 processes photo, video, and audio annotationsprocessing posts and notifying moderators of certain posts, as well asmanaging user replies. In some embodiments, the media module alsoprocesses video annotations by capturing and presenting a particularframe (such as the first frame) as a thumbnail image representing thevideo in the annotations portion of the display.

The content serialization engine 177 interfaces with the database 135 tolock content, update content, and otherwise process user annotations.The CSE 177 facilitates content delivery among multiple users. Forexample, when a first user provides an annotation regarding a particularpage of content, in some embodiments, synchronization module 165 locksthat page and prevents access to the page by other users until theannotation process is complete. In some embodiments, the CSE 177maintains a queue of new annotations and processes annotations bycreating new content pages and media pages containing the newannotations as further described herein.

The statistics module 183 generally tracks data related to postedannotations. For example, in some embodiments, the statistics module 183tracks the number of annotations posted for each page in a givendocument and presents an indication of which page has the most number ofnew posts or a certain number of posts within a given period of time,such as in memory book mode as further described herein.

The print module 189 is generally directed to printing or otherwiseoutputting content according to user inputs and preferences. Forexample, the print module 189 creates PDF files or other document filesfor versions of content output such as dynamic print and memory bookcreation as further described herein.

The conversion module 185 is generally responsible for processing andformatting raw original content for use by the system and for users toannotate. For example, the conversion module parses original contentinto paragraphs, formats the content for presentation, and createsbookmark IDs or other identifiers for each paragraph used by the CSE 177to create new pages when annotations are added as further describedherein.

The ecommerce module 187 processes payments and generally handlesmonetary transactions associated with use of the system. For example,the ecommerce module manages shopping carts and other purchase vehicles,processes credit card payments and other payments, and also interfaceswith other modules such as integration modules including the paymentgateway 197 and external print services 199.

The personalization module 191 and the profile module 193 are generallyresponsible for processing inputs regarding user accounts. For example,the profile module 193 processes user administrative requests regardingpassword and address changes. The personalization module 191, sometimesin conjunction with the profile module 193, handles other inputs such asassociating a personal photo or icon to present next to user postings orin a user's business card, as well as other general information aboutthe user such as hobbies, favorite websites, etc.

The search module 195 is generally responsible for indexing andprocessing search operations on both original text and on annotations.For example, search module 195 allows users to search not only documentcontent, but also annotations provided by other users and otherinformation. Users can search for annotations provided by a particularuser, for a particular text string contained in annotations, and inputother search expressions to locate information.

The system also includes various modules, such as a payment gatewaymodule 197 and a print services module 199, for integration withexternal or third-party systems. For example, in some embodiments thepayment gateway module 197 provides an interface to process all or partof the payments using a third-party payment provider. In otherembodiments, the print services module 199 provides an interface forprinting special jobs, such as hardcover book binding or other types ofbook creation of content, using a third-party or other external printservices provider.

The business tier also includes a commons module 201. The commons modulegenerally includes a utility library of various APIs and other systemcalls used for interfacing with the operating system, hardwarecomponents, the data store 135, modules in the various other tiers, etc.

FIG. 3 is a flow chart of a method to synchronously navigate sharedannotations according to an embodiment of the invention. The systemreceives an annotation generated by a first user at a first client, step230. For example, the system receives a text comment related to adocument or a picture related to the document. The annotation isassociated with a first indication, step 235. For example, theannotation may be associated with an icon or other indication embeddedin the document. The system receives input from a second user at asecond client to navigate a first portion of a display at the secondclient, step 240. The navigation input causes the first indication to bedisplayed in the first portion of the display at the second client. Inresponse to the input, the system automatically displays the annotationin the second portion of the display at the second client, step 245.

Conversely, the system also processes and navigation inputs navigatingthe second portion of the display. Thus, the system also can receive aninput from a second user at a second computer to navigate a secondportion of a display at the second client. The navigation input causesthe annotation to be displayed in the second portion of the display atthe second client and, in response to the input, the systemautomatically displays the first indication in the first portion of thedisplay at the second client.

In some embodiments, the system divides content into a plurality ofpages. Thus, a book might be divided into chapters and each chapterformatted as a particular HTML or other similarly encoded page. Thesystem loads an entire page of original content into the first portionof the display and also the entire page of related annotations for thepage in the second portion of the display. In some embodiments, thesystem first loads only those annotations corresponding to indicationsimmediately displayed upon loading the page into the first portion andthen loads annotations corresponding to off-screen indications whichachieves, among other benefits, a performance boost in terms of loadtimes. As further described herein, code such as a JavaScriptsynchronization module monitors user navigation inputs and mouse inputsand states to determine whether and when to synchronously scroll orotherwise display indications and their related annotations in the firstand second portions of the display.

For example, a JavaScript event or other similar program code returnsidentifiers corresponding to indications that are visibly displayed inthe first portion. In some embodiments, the system employs a namingconvention correlating indications with annotations. Thus an indicationlabeled I1 would have its corresponding annotation labeled A1 and anindication with an identifier of I2 would have its correspondingannotation identified as A2, etc.

When a navigation in put is received relating to the first portion ofthe display, the system determines, based on the JavaScript event dataand indication identifiers, any indications visible in the first portionof the display and then automatically executes JavaScript code or otherprogram code to display, in the second portion of the display, theircorresponding annotations according to the naming convention. Forexample, if the system identifies indication I1 as visible in the firstportion of the display, then it automatically executes code to displayA1 in the second portion of the display. In some embodiments, when anumber of indications are visible in the first portion of the displayand there is insufficient screen space in the second portion of thedisplay to display all of the corresponding annotations, the system,starting with the first indication displayed, displays as manycorresponding annotations as possible in the second portion of thedisplay.

In some embodiments, when a navigation input is received related to thesecond portion of the display, the system determines, based on theJavaScript event data and annotation identifiers, any annotationsvisible in the second portion of the display. The system also determinesany indications visible in the first portion of the display aspreviously described herein. If the first portion of the display alreadyshows the indication corresponding to the first annotation appearing inthe second portion of the display, then the system does not redraw thescreen. If the corresponding indication in the first portion is notdisplayed, then the system executes JavaScript code or other programcode to display, in the first portion of the display, the indicationcorresponding to the first annotation appearing in the second portion ofthe display according to the naming convention. For example, if thesystem identifies annotation A1 as visible in the second portion of thedisplay, then it checks if indication I1 is visible in the first portionof the display. If the indication is not visible, the systemautomatically executes code to display I1 in the second portion of thedisplay.

Thus users at a plurality of clients are able to view content, as wellas collaboratively annotate and view annotations provided by otherusers. For example, several users may negotiate a contract by sharingfeedback and other annotations to produce a final version of thecontract. The annotations would later serve as a record of positionsregarding various clauses of the contract, how the document was created,who was in favor of various positions, etc. In some embodiments, thesystem also provides user authentication and secure access to content,allowing only a limited number of authorized users to access and/orannotate content. Thus, adverse parties are presented with a securespace in which they can collaboratively and synchronously annotatecontent. As another example, a school might post a number of photographscontaining unidentified subjects. The system would provide a means forregistered alumni or other parties to identify the subjects for theschool archives, etc.

FIG. 4A presents a block diagram of an exemplary screen display of ashared annotation system according to an embodiment of the invention. Adisplay 250, such as a browser display or other software applicationdisplay, is divided into a first portion of 252 and a second portion254. The first portion 252 contains information content provided by aserver and the second portion 254 contains annotations related to thecontent in the first portion.

A user requests content from the content server and the content isdelivered via the network to the user at a client and presented in thefirst portion of the display 252. Content may include, for example, thetext of a book, graphical content such as a picture album or photoalbum, a proposed legal document or business agreement, multimediacontent, or other types of content. For example, the text of a bookappears in the first portion 252 of the display 250. Indicationsassociated with user annotations are embedded within the content of thefirst portion of the display. Thus, indication 256 corresponding to userannotation 262 and indication 258 corresponding to user annotation 264are embedded in the content of the first portion 252. The actualannotations 262 and 264 are presented in the second portion of thedisplay 254.

In some embodiments the display 250 also includes a third portion 260including additional references to indications contained in the firstportion 252. For example, as shown, additional indications 266, and 268corresponding to indications 256 and 258 are presented in a thirdportion of the display 260. Users can scan the third portion of thedisplay 260 to quickly determine whether indications exist in thecontent presented in the first portion of the display 252.

The system also presents navigation interfaces such as scroll bars 272and 274, as well as a menu bar 276 at the bottom of the display 250which provides users with an interface to navigate a document dividedinto chapters/sections or jump to additional pages, etc. The system alsopresents standard interface elements such as final, edit, view,favorites, tools and help menus 278 as known in the art and common inInternet browsers.

In addition, the system presents a plurality of icons 280 designed toprovide an interface for common operations that users might want toperform when viewing content such as a document, a photo album, or abook. Icons presented allow users to zoom in, zoom out, add a comment orannotation at a specific location within the content, highlight aspecified region within the content, annotate a picture for a specifiedlocation, annotate video for a specified location, annotate audio for aspecified location, create or interact with a discussion group relatedto the content at a specified location, perform a search, or resize theportions of the display.

FIG. 4B presents a block diagram of two exemplary screen displays of ashared annotation system according to one embodiment of the invention.The two screen displays 282 and 300 show versions of the same display attwo different points in time. The display is divided into a firstportion 284 and a second portion 286. The first portion contains contentas well as indications 288 and 290 associated with user annotations 292and 294 respectively. Navigation means, such as scroll bars 296 and 298,are also provided.

As previously described, a user navigating the display 282, for example,by using slider 296, would cause the display 282 to change as shown in asecond screen display 300 of the same display at a later point in timeafter the system processes the navigation input. The user scrolls thecontent in the first portion 284 such that indication 288 disappearsfrom the first portion 284 and indication 302 appears. Similarly,annotation 292 associated with indication 288 automatically disappearsin the second portion 286 of the display 300 and annotation 304corresponding to indication 302 automatically appears in the secondportion 286. As previously discussed, the system also conversely scrollscontent in the first portion 284 of the display 282 when a usernavigates content in the second portion 286 of the display 282. Forexample, the system automatically scrolls content in the first portion284 of the display 282 according to a user input, such as a scroll barslider 298 or other similar means, to navigate annotations in the secondportion 286 of the display 282. Thus, an indication 288 corresponding toan annotation 292 in the second portion 286 of the display 282 wouldautomatically appear or disappear in the first portion 284 of thedisplay 282 when the corresponding annotation 292 appears or disappearsin the second portion 286 of the display 282 according to a usernavigation input.

FIG. 5 presents a flow chart of a method for processing an annotationaccording to an embodiment of the present invention. A user selectscontent via a selection tool or other means, step 330. For example, auser might employ a text tool to highlight and select several words inthe text of a document which the user wishes to annotate with a textualcomment, an uploaded picture, a video, a sound recording, etc.JavaScript event code or other program code related to mouse inputs andother user inputs captures various metadata regarding the userselection. For example, the event code captures and returns a uniqueparagraph identifier tag, a starting point value or offset (incharacters from the start of the identified paragraph, pixels, or othermetrics known in the art), and ending point value or offset.

While the example discussed herein with respect to FIG. 5 relates toprocessing a text selection, those skilled in the art will recognizethat the process could similarly apply to selecting other forms ofmultimedia content including pictures, video, etc. For example, in oneembodiment a user can crop one or more areas of a picture the userdesires to annotate. For example, a user could crop a single area of apicture for an annotation or a user could crop several different (oroverlapping) areas of the same picture for several differentannotations. The user selects the area using a rectangular croppingtool. The system captures the x,y coordinates of the corners of therectangle to create a mapping or overlay representing the selection ofthe original image. Once the image area is selected the user may alsoassign additional attributes to the selection (such as a person name, aproduct identifier, a price, a location, a theme, a date, etc.). In someembodiments, users may also indicate a frame or other location in avideo using similar selection means for individual frames of a video.

The system expands the selection to an appropriate level of granularity,step 335. A user might select several letters of a word and the systemmight expand the selection by highlighting the entire word. In someembodiments, for example to preserve system resources or to limitannotations from cluttering a screen or for other design-relatedconsiderations or specified goals, the system imposes a pre-set limit onthe ability of a user to annotate text to a certain level ofgranularity. Thus a user may only be able to annotate whole words oronly words at the end of a sentence. For example, if a user were able toannotate every individual letter of words in a text, a single word sucha “Kennedy” might have as many as seven distinct indications(corresponding to the total number of letters in the word) presentedwith the word. This would likely render display of content in the firstportion of the display extremely cumbersome and severely limit theability of the system to efficiently present information to users.

Similarly, the system may also limit the number of indications presentedrelated to particular sections of text or other content. Indications maybe consolidated or combined in the interest of making content morereadable, visually comprehensive, or otherwise accessible. For example,annotations provided by four different users might be associated with asingle indication embedded in the content and displayed in the firstportion of the display rather than with four separate indications in thefirst portion. In the second portion, however, each individualannotation provided would automatically be displayed when itscorresponding indication is presented in the first portion of thedisplay.

After the user selects the desired content, the user indicates itsdesire to post an annotation related to the selected content, step 340.For example, a user may select a section of text and then click a “post”button or icon. The system presents a form or other similar inputmechanism, step 345, which allows the user to input and submit/uploadthe desired annotation to the content server, step 350. For example, aform window may open allowing the user to input a text annotation or atree-view directory structure may be presented allowing the user toselect a file (such as a picture, a video, an audio clip, etc.) toupload as an annotation.

The annotation input by the user and any related metadata are thenuploaded via the network to the content server and stored in the datastore for further processing, step 355. The system generallycommunicates metadata indicating, among other things, the desiredposition of the annotation within the content of the first portion, theuser's identity, the type of annotation, etc. For example, in oneembodiment, JavaScript code captures the events of a mouse clickindicating the beginning of a selection, mouse drag changing the x,ycoordinates for the selection, and a mouse up or un-click ending theselection. This data is saved into an HTML form attribute andtransmitted to the server when the form is submitted. In someembodiments, as further described herein, the system also indicates theposition of a desired annotation by providing metadata indicating anoffset from a particular starting point within the document content anda selection length corresponding to the user selection of steps 330 and335. For example, if a user selects text several sentences into aparagraph or other arbitrary section of a document, the system maycommunicate metadata indicating, from the start of the paragraph orother section, an offset corresponding to the number of characters atwhich the annotation begins and a length corresponding to the number ofcharacters selected for the annotation.

In some embodiments, the system uses a content serialization engine(“CSE”) or other similar means to lock the page of the document to whichthe annotation relates, step 360. In some embodiments, this preventsmultiple CSEs from accessing and updating the page at the same time. Forexample, in a parallel processing environment or other environmentsupporting multiple CSEs in the same system, each CSE locks anindividual page prior to updating the page to prevent other CSEs fromaccessing and simultaneously updating the page which would createproblems such as content synchronization, etc. In some embodiments, theCSE lock also prevents other users from requesting the page from thecontent server while the system is processing the user's submittedannotation and embedding a related indication in the page of thedocument.

As discussed, the system parses the metadata associated with theannotation, step 365. Using the length, offset, and other data providedwith the metadata, the system determines a location in the documentcontent at which to embed an indication corresponding to the annotation.The system then recreates the original page (including any additionalpages created by the annotation) to embed an indication corresponding tothe annotation, step 370, and updates the database with the new page,step 371. In some embodiments, the system replaces the old page storedin the database with the new page. In other embodiments, the old page isretained in order to track document versions and related annotations.The CSE lock is removed, step 372 and users at other clients are thenable to request, retrieve, and view the new page containing the newindication corresponding to the new annotation.

FIG. 5A presents an exemplary sample of code for an XHTML formatted pageof content containing an indication corresponding to an annotation whichwould be presented in the first portion of a display according to oneembodiment of the invention.

The code sample uses various XHTML elements such as Div elements, Spanelements, Highlight elements, and Content elements to present thecontent and corresponding indication.

Div element class shrdbk_main 373 is a div element that wraps the wholebook text. In some embodiments, this element is used in a non-embeddedmode to separate the indications or book items icons from the pagetext/content. Thus, a user would be able to toggle presentation ofcontent both with and without indications being displayed.

The system also uses a number of different types of Span elements. Spanelements are tags generally used to group inline elements in a document.Span element shrdbk_start_element 374 is span element that is used as anindicator for the start location of the related text of the book item.The id attribute contains the type of the book item or indication (‘C’for comment, ‘I’ for image, ‘A’ for audio and ‘V’ for video), anidentifier for the indication, and a starting location of this elementin a numerical representation corresponding to a number of characters orother metric (e.g._554). The indication identifier is used in varyingembodiments to distinguish between indications and also to assist incontent navigation, for example if a user wishes to jump to the nextindication, etc.

Span element shrdbk_end_element 375 is a span element that is used as anindicator for the end location of the related text of the book item orindicator. The id attribute contains the type of the book item, the bookitem id, and a location or offset of this element in a numericalrepresentation (e.g._681).

Span element shrdbk_icons 376 is a span element that contains the imageof the icon or indication to be embedded. For each location in thecontent, such as the book text, a different type of indication icon isused to represent each different type of annotation (e.g.—textannotation, multimedia annotation, etc.). The image element that isincluded for the indication represents the type of the items and theindex number of the first item at this location, according to itsappearance order within the book text.

Highlights Div elements idYellow, idFirstLine, and idLastLine 377 are aset of div elements that are used for highlighting the related textcorresponding to the annotation. For example, when a book item isselected, by clicking on its title, the text range that represents therelated text is located according to the start and end span elements.Text rectangles are created from the given text range and these divelements positions are set according to the text rectangles.

For each shrdbk_icons span element there is also a corresponding divelement, Content Div 378, which includes a representation of each of theitem(s) that the span element contains for the specific location. Thisdiv element is generally displayed when the mouse cursor is over theimage icon. The div element contains links for the related text of eachof the book items and when clicking on those links the related text ishighlighted. In some embodiments, another role of those links is tosynchronize the media area/first portion with the current viewed item.Thus, when the user clicks on one of the links except from thehighlighting of the related text, the media area automatically scrollsto the appropriate item in the second portion of the screen. In someembodiments, if the current displayed items have a different type fromthe item that was clicked, the type of the viewed media is changed tothe equivalent type according to clicked link.

FIG. 5B presents an exemplary sample of code for an XHTML formatted pageof content containing an indication corresponding to an annotation whichwould be presented in the first portion of a display according to oneembodiment of the invention.

The code sample also uses various XHTML elements such as Div elements,Span elements, Highlight elements, and Content elements to present thecontent and corresponding indication. More specifically, the code sampleprovides exemplary span elements for presenting content in embedded andnon-embedded modes.

For example, span elements 379 are used for displaying icons and othercontent in non-embedded mode. The element at the beginning of theparagraph is used as an anchor for the book item icon. The contentelement is placed in the bottom of the HTML document and includes thebook item title as well as any relevant functionality.

Span elements 380 are used for displaying icons and other content inembedded mode. Here, the element within the paragraph is used as ananchor for the book item icon. The content element is similarly placedin the bottom of the HTML document and includes the book item title aswell as any relevant functionality.

In addition, Span elements 381 present exemplary uses of span elementsas start and end anchors for highlighting selected or annotated content.

FIG. 6 presents a flow chart of a method of annotating a visual elementaccording to an embodiment of the present invention. In someembodiments, users may wish to provide annotations corresponding tovisual elements such as pictures or video clips. Thus, the user views avisual element, such as a picture, step 385, and selects a pictureelement to annotate, step 387. As discussed, in some embodiments, theuser might use a selection tool to crop or otherwise select pictureelements, for example, by drawing a box around or otherwise selecting aperson in a photo.

The system then presents an annotation form or other input means, step389, and the user inputs and submits the annotation, step 391. In someembodiments, the system allows the user to submit multiple annotationsfor a single picture. Thus, in these embodiments, control may return tostep 387 for the user to select additional picture elements. Forexample, the user may select a first element and input an annotation forthe first element and then select a second element and input a secondannotation for the second element, etc. The annotation(s) are thenuploaded via the network to a content server and stored in a data storewhere they are associated with the visual element(s), step 393. In someembodiments, the system also maintains and updates an index ofannotations corresponding to visual elements, step 395. For example,users may provide annotations identifying subjects in visual elementsand the system maintains an index of identified subjectscross-referenced with their corresponding visual elements. Using searchmeans known in the art, users could access such an index to locate allvisual elements in a content document, such as a photo album, a book,etc., in which a particular subject appears. The content serializationengine then locks the page, embeds any required indications in theoriginal content as previously described herein, and updates theoriginal page in the data store as previously described herein, step397.

FIG. 6A presents a flow chart of a method of recreating a page ofcontent according an embodiment of the invention. Once the contentserialization engine locks the page, the system retrieves the existingpage from the data store and a list of all related annotation to contentand indications of the page, step 399. The system determines if anyelements of the page remain to be processed, step 401. For example,content generally comprises various XML tag elements corresponding touser selections and other content related to annotations. In oneembodiment, the CSE organizes elements into a list corresponding totheir location on the page. If no further elements remain to beprocessed, control proceeds to step 408 and the routine ends.

Otherwise, the system determines whether the next element in the list isassociated with a content identifier, step 403. For example, in oneembodiment, the system determines whether the element has a sharedbk XMLtag identifier. If the element does not have an identifier, then it isgenerally not associated with an annotation and recreation of theelement is generally not required and thus control passes to step 407and the system proceeds to process the next element in the list.

Otherwise, the system checks for annotations related to the element,step 405, and recreates the element, step 406 as further describedherein. For example, in some embodiments, elements associated withannotations are associated with unique content identifiers. Thus, anelement and all its related annotations might share the same or relatedcontent identifiers according to embodiments of the invention. After thesystem determines which annotations relate to the current element, thesystem recreates the element, step 406, inserting any necessaryindications, rollovers, or other items as further described herein.

FIG. 6B presents a flow chart of a method of processing an elementduring page creation according to an embodiment of the invention. Afterthe system determines that an element should be recreated (or in someembodiments originally created), the system orders all annotationsassociated with the element into a list according to their location,step 409. Thus, for a particular sentence, paragraph, page, etc. thesystem creates an ordered list of all annotations using the offsets andlocation metadata stored with the annotations. If no further annotationsremain to be processed, control passes to step 417 and the routineexits. Otherwise, the system processes the location metadata associatedwith the annotation to determine the location in the first portion ofthe display to place an indication or icon corresponding to theannotation in the second portion of the display, step 411.

The system processes the annotations in the list to determine whetherthere are multiple annotations associated with the same location, step412. If there are multiple annotations, then the CSE creates the XHTMLcode or other code, inserting a multiple annotation indication or icon,step 413. If there are not multiple annotations, then the CSE createsthe XHTML code or other code, inserting a single annotation indicationor icon, step 414. For example, in some embodiments, certain indicationsindicate that they correspond only to a single annotation. An imageindication corresponds to an image annotation, an audio indicationcorresponds to an audio annotation, etc. In other embodiments, ifmultiple annotations are made at the same location in the originalcontent in the first portion of the display, the system embeds orotherwise places a multiple annotation indication which indicates thatmore than one annotation has been made at a particular place in theoriginal content.

In some embodiments, the CSE also creates XHTML code or other code,generating a rollover action associated with the indication, step 415.For example, the CSE engine retrieves metadata associated with theannotation(s) for a particular location and indication which lists atitle for the annotation, the annotation's author(s), etc. The systemthen proceeds to process the next element, step 416 and control returnsto step 410.

FIG. 7 presents a flow chart of a method of providing a customizeddocument related to a shared annotation system according to anembodiment of the present invention. A user may view a book on homerepair in which the main document content of the book provides chapterson framing, wiring, plumbing, etc. Within each chapter, other users mayhave provided annotations related to various tasks described, etc. Oneuser might indicate a particular brand of pipe that they found useful incompleting a certain project or a particular type of light fixturewell-suited to applications. Another user might provide additionalphotographs of their project with additional text comments, etc. tosupplement the information of the original book. Thus, users may wish toview and wish to purchase or otherwise obtain customized documents,including these related annotations and other items such as toolsrequired to complete certain projects, etc.

For example, as shown in FIG. 8, a block diagram of a sample page 455from a customized document according to an embodiment of the presentinvention is presented. The sample page 455 includes the documentcontent 460 corresponding to the content of the document presented inthe first portion of the display. In some embodiments, the page 455 alsoincludes annotations and other comments related to the document content460 such as textual annotations 465, picture annotations 466, audio orvideo annotations 467, annotations related to discussion group content468, advertisements 469, links to related merchandise 470, and otherinformation. Those skilled in the art will appreciate that thisinformation could be presented in a variety of manners or layouts. Forexample, as shown in FIG. 8, the document content 460 is centrallydisplayed and surrounded by related annotations including callouts toindications contained in the content 460 and other visual cues.

Thus, returning to FIG. 7, the user selects a particular book edition,step 420. A user may select among a number of different books ordocuments containing content related to a desired subject or a user mayonly select certain chapters within a book. For example, a user mayconsult a home repair manual, but only be interested in the chapter onplumbing or on wiring and not wish to be provided with the entire book.

The user also determines and selects annotations they wish provided withtheir customized document/book, step 425. A user may wish to be providedwith all annotations related to the desired content, only annotationsauthored by an arbitrary/particular user, only a specific annotationcontaining certain information the user finds useful, such assupplemental photos, video, other types of annotations, etc. In someembodiments, the system also offers the user a promotion or other offerassociated with the content and the user determines whether or not toaccept the promotion, step 430. Thus, a user purchasing a home repairmanual chapter related to dry walling might also be presented with theoption to purchase items and merchandise related to the project such ashammers, nails, screws, plaster, tape, drywall, or even other books orinformation related to the project. For example, the system may alsooffer a video of how to complete a sample project for an additionalpremium.

If the user accepts the promotion or offer, the user selects the relatedmerchandise or otherwise complies with responding to and accepting theoffer, step 435. Otherwise, control passes directly to step 440 and theuser selects a particular format for the customized document. Forexample, a user may wish a hardcopy paper version of a customizeddocument or they may prefer to receive the document electronically orsome combination thereof. As necessary, the user also selects a deliverymethod, such as via mail, express mail, download, etc., step 445. Theuser also inputs any necessary payment information, personalinformation, registration information, license information, or otherinformation required to complete and process the transaction, step 450.

FIG. 8A presents a screenshot of an exemplary article page of a memorybook according to an embodiment of the present invention. A memory bookgenerally comprises a customized printout of content and relatedannotations. In some embodiments, memory books are compiled and boundaccording to user preferences.

For example, in some embodiments, users create a memory book bycustomizing existing content provided by content creators. Thus, forexample, a content provider might use the system to post an originalarticle to the Web containing text, photos, and other multimediaelements recounting or otherwise related to an event such as a HarleyDavidson rally or a Britney Spears concert. The original article alsogenerally contains indications and corresponding annotations input byvarious users responding to the original article. A user can then createany number of custom memory books from the original article by uploadingadditional multimedia elements and selecting specific annotations toinclude in their personal memory book. For example, a user attending theHarley Davidson rally can create a memory book containing photos,annotations, and other elements related to that user's own personalexperience at the Harley Davidson rally. As another example, a userattending the Britney Spears concert creates a memory book related totheir own personal concert experience with their own photos from beforethe show, after the show, photos from during the show, relatedannotations, the user's own textual inputs, etc.

For example, a user who went to the Harley Davidson rally uploads theirown pictures taken at the rally to replace or supplement the pictures inthe original article posted by the content provider. In someembodiments, a user also uses pictures posted as annotations by otherusers to replace or supplement pictures of the original article or theyuse additional pictures provided by the content provider or othercontent providers.

Users also select custom annotations to include with the memory book byfiltering or otherwise selecting annotations from the set of annotationsposted by other users regarding the original article. In one embodiment,a user automatically selects annotation from a list of friends who postannotations. In other embodiments, users select annotations individuallyor based on criteria such as ratings from other users, annotation type,etc.

Thus, a user creates their own personal memory book from the originalarticle. The personal memory book generally contains the text and othercontent of the original article including additional pictures, text,videos, and related annotations selected or otherwise input by the user.As further described herein, the user then has the option to print outthe memory book and have it bound or otherwise preserved, for example asa souvenir.

An article page of a memory book generally includes article text of theoriginal content along with embedded photos with captions, embeddedindications, and other items as further described herein. Generally, thepresentation of the article page is formatted as closely as possible tothe view a user would be presented with online. In some embodiments,however, the pagination is different since the content is now beingproduced on a printed page as opposed to on a display. Indications andother content elements, however, are generally presented in the samelocation within the content as they are presented in a display, thusenabling users to quickly reference between online and printed versions.

The article page includes one or more of the following: a header 471,embedded images 472, image captions 473, embedded icons or indications474, and a footer 475. The header 471 generally remains consistentacross pages throughout a memory book, thus unifying contentpresentation, etc. In some embodiments, the header includes a graphic,such as a logo, and heading text which may be used by the system tocreate a table of contents, an index, etc. Embedded images 472 includeimages originally presented in the original content as well as imagesselected by a user for inclusion in the memory book. For example, a usercreating a memory book of a trip might select only particular photosfrom a set of photos for inclusion within the memory book. In someembodiments, images 472 also contain an image caption 473 which mayinclude the poster's username, the date the photo was posted, a titlefor the image 472, etc. Embedded icons or indications 474 generallyappear in the same location of the content as they do when presented ina display. In some embodiments, however, icons 474 are renumbered foreach individual page (e.g.—starting from 1 for the first indication 474on each page) and thus the numbering scheme for indications 474 maydiffer from the online version of the book. In some embodiments, thearticle page also contains a footer 475 containing the book's title,page number, publisher information, etc.

FIG. 8B presents a screenshot of an exemplary comments page of a memorybook according to an embodiment of the present invention. The commentspage of a memory book generally includes comments and other annotationsinput by users online and generally is included on one or more separatepages falling after the article page as opposed to on the same page asthe article text itself. The article comments page includes one or moreof the following: a header 476, a sub-header 477, a comment or replyicon 478, a comment title 479, a username and date of post 480, acomment text or other annotation content 481, one or more replies 482,and comments by various types of members 483.

The header 476 of the article comments page is generally a graphic andcorresponds to the header of the article page of the memory book.Sub-headers 477 indicate the printed page in the memory book whichcontains the article to which the annotations are related. Comments orreply icons 478 are generally graphics indicating a type of comment. Forexample, a text comment might have a balloon with text in it as an icon478 and an audio comment might have a musical note as an icon 478.Comment titles 479 indicate any heading a user inputs to associate withtheir comment. In some embodiments, comment titles are printed indifferent colors according to the type of user. For example, comments byregular members might be printed in black, comments by moderators 482 inred, etc. In some embodiments, comment text 481 is also displayed invarying colors according to user types. In some embodiments, a usernameand date of post 480 are also displayed for each annotation. Replies 482associated with comments may also be presented.

FIG. 8C presents a screenshot of an exemplary dynamic print pageaccording to an embodiment of the invention. Dynamic print pages aregenerally formatted to include comments and other annotations just belowthe text to which they refer. As shown, the page includes the originaltext 484 including inline indications corresponding to the first portionof the display. The page also includes annotations such as textcomments, images, etc. as would be presented online in the secondportion of the display.

FIG. 9 shows a method of presenting a selected multimedia element whilenavigating a document in shared annotation system according to anembodiment of the present invention. When viewing a multi-page documentor viewing several documents, users may wish to visually retainpresentation of a multimedia element, such as a chart, a table, apicture, etc. from one page while viewing content on another differentpage. For example, a user viewing several pages of a document related toa particular company's financial outlook might find it useful to retaina chart of the stock price or a table of pro form a income projectionsfrom one page while viewing information on a second page. In someembodiments, the system achieves the goal by allowing users to select amultimedia element and then floating the selected element on top of orintegrating the selected element with subsequent pages that are viewed.

Thus, the user selects a multimedia element in a first page, such as apicture, using various input means previously described herein, step495. The selected element is identified in the content database, step500, and floated or otherwise displayed in the browser window, step 505.For example, the user client communicates the selected elementidentifier to the content server which retrieves another instance of theelement and floats the element in the browser window containing theoriginal content or displays the selected element in a new window orframe. In some embodiments, the system recreates the first page,removing the selected element and floats or otherwise displays theselected element over the location in the content where the selectedelement previously resided. In other embodiments, the system does notimmediately float or otherwise display the selected element, but insteadonly identifies the selected element and only floats the selectedelement when the system receives input to navigate to a second page,step 510. In some embodiments, the system retrieves the original versionof the second page stored in the database, step 515, and creates a newsecond page to display by modifying the second page and embedding theselected element from the first page, step 520. The modified second pageis then presented with the original second page content now includingthe selected element, step 525.

Systems and modules described herein may comprise software, firmware,hardware, or any combination(s) of software, firmware, or hardwaresuitable for the purposes described herein. Software and other modulesmay reside on servers, workstations, personal computers, computerizedtablets, PDAs, and other devices suitable for the purposes describedherein. Software and other modules may be accessible via local memory,via a network, via a browser or other application in an ASP context, orvia other means suitable for the purposes described herein. Datastructures described herein may comprise computer files, variables,programming arrays, programming structures, or any electronicinformation storage schemes or methods, or any combinations thereof,suitable for the purposes described herein. User interface elementsdescribed herein may comprise elements from graphical user interfaces,command line interfaces, and other interfaces suitable for the purposesdescribed herein. Screenshots presented and described herein can bedisplayed differently as known in the art to input, access, change,manipulate, modify, alter, and work with information.

While the invention has been described and illustrated in connectionwith preferred embodiments, many variations and modifications as will beevident to those skilled in this art may be made without departing fromthe spirit and scope of the invention, and the invention is thus not tobe limited to the precise details of methodology or construction setforth above as such variations and modification are intended to beincluded within the scope of the invention.

1. A method for automatically navigating a document in a web page, themethod comprising: separating a web page into at least a first displayPortion and a second display portion; displaying in the first displayportion of the web page a document along with annotation indications;receiving multiple annotations related to the document, the annotationsgenerated by different users with different user computers, theannotations displayed in the second display portion of the web page;associating the annotations with the annotation indications in thedocument displayed in the first display portion of the web page; andreceiving, from a user, an input to navigate to one of the annotationsin the second display portion, and in response to the input,automatically scrolling the document in the first display portion theinput causing the annotation indication associated with the annotationto be displayed in the first display portion of the web page.
 2. Themethod of claim 1, further comprising: allowing different users tosubmit different annotations wherein the different annotations areassociated with related portions of the document; and combining theannotation indications in the first display Portion of the web page thatare associated with the related portions of the document such that thecombined annotation indications correspond to the different annotationsdisplayed in the second display portion of the web page.
 3. The methodof claim 2, wherein the combining of the annotation indications overlapsthe annotation indications.
 4. The method of claim 1, wherein thedocument comprises an electronic book.
 5. The method of claim 1, whereinthe document comprises a digital photo album containing one or moredigital photos.
 6. The method of claim 1, further comprising selectivelyfiltering the annotations based on user input.
 7. The method of claim 6,wherein selectively filtering the annotations is based upon at least oneof the group consisting of a list of user friends, individual userselection, ratings from other users, and annotation type.
 8. The methodof claim 1, wherein the document comprises a multimedia document.
 9. Themethod of claim 1, wherein the annotation comprises a text annotation.10. The method of claim 9, wherein the text annotation comprises acomment related to the document.
 11. The method of claim 1, wherein theannotation comprises a graphical annotation.
 12. The method of claim 11,wherein the graphical annotation comprises a photograph.
 13. The methodof claim 1, wherein the annotation comprises an audio annotation. 14.The method of claim 1, wherein the annotation comprises a videoannotation.
 15. The method of claim 1, wherein the annotation comprisesa multimedia annotation.
 16. The method of claim 1, wherein theannotation comprises a discussion group related to the document.
 17. Themethod of claim 1, wherein the input comprises an input to scroll thefirst display portion of the web page.
 18. The method of claim 1,wherein the input comprises an input to navigate to a portion of thedocument containing the first indication.
 19. The method of claim 1,wherein the first indication comprises a graphical indication.
 20. Themethod of claim 1, wherein the first indication comprises an icon. 21.The method of claim 1, wherein receiving an annotation comprisesreceiving form data submitted by the user at a first user computer. 22.The method of claim 21, wherein receiving form data comprises receivingHTML form data.
 23. The method of claim 1, wherein associating one ormore annotations with one of the annotation indications in the documentcomprises: identifying a portion of the document to which the annotationrelates; and associating the annotation indication with the portion ofthe document to which the annotation relates.
 24. The method of claim23, wherein the annotation comprises a discussion group related to theportion of the document.
 25. The method of claim 23, the method furthercomprising adding the annotation to a data structure stored in memory,the data structure comprising a list of annotations relating to portionsof one or more documents.
 26. The method of claim 25, wherein the listof annotations comprises a list of bookmarks.
 27. The method of claim26, wherein selecting an annotation from the list of bookmarks displays,in the first display portion of the web page, at least a portion of adocument to which the annotation is related and displays at least theselected annotation in the second display portion of the web page. 28.The method of claim 23, wherein associating the annotation indicationcomprises embedding the annotation indication in the portion of thedocument to which the annotation relates.
 29. The method of claim 28,wherein embedding the annotation indication comprises: receivinglocation data related to the portion of the document; processing thelocation data to determine a first location within the document relativeto a location of the portion within the document; and generating a newversion of the document, the new version of the document containing theannotation indication embedded at the first location.
 30. The method ofclaim 29, wherein the location data comprises on or more from the groupcomprising: a document identifier, a section identifier, a chapteridentifier, a bookmark identifier, a portion length, and a portionoffset.
 31. The method of claim 29, the method further comprisingreplacing a first version of the document stored in memory with the newversion of the document.
 32. The method of claim 31, wherein replacing afirst version of the document comprises overwriting a first version ofthe document.
 33. The method of claim 1, wherein receiving an annotationcomprises receiving an annotation related to an image contained in thedocument.
 34. The method of claim 33, wherein receiving an annotationrelated to an image comprises receiving information identifying one ormore subjects of the image.
 35. The method of claim 34, the methodfurther comprising associating the one or more subjects with the image.36. The method of claim 35, wherein associating the one or more subjectswith the image comprises updating a data structure stored in memory, thedata structure storing associations between one or more images and oneor more subjects of the one or more images.
 37. The method of claim 1,wherein the annotation comprises a commercial offer.
 38. The method ofclaim 37, wherein the commercial offer comprises an offer to purchase aproduct related to the document.
 39. The method of claim 38, the methodfurther comprising processing a request by a user at a client topurchase the product.
 40. The method of claim 39, the method furthercomprising transmitting the product and the document to the user. 41.The method of claim 1, the method further comprising communicating, to auser at a client, an offer to purchase the document and a set ofannotations related to the document.
 42. The method of claim 41, whereinthe set of annotations related to the document comprises a set ofannotations selected by the user.
 43. The method of claim 41, the methodfurther comprising processing a user request to purchase the documentand the set of annotations.
 44. The method of claim 43, whereinprocessing the user request comprises printing the document and set ofannotations.
 45. The method of claim 44, comprising, for each annotationrelated to a portion of the document, printing the annotation and therelated portion of the document on the same page.
 46. The method ofclaim 43, wherein processing the user request comprise transmitting thedocument and the set of annotations to the user.
 47. The method of claim1, the method further comprising: authenticating the user at a firstclient and authorizing the user at the first client to provide theannotation; and authenticating the user at the second client andauthorizing the user at the second client to navigate the document. 48.A method for annotating a document located at a first server accessibleby multiple users at remote clients, the method comprising: separating aweb page into at least a first display portion and a second displayportion; displaying in the first display portion of the web page adocument along with annotation indications; receiving at the firstserver annotation data related to the document, the annotation datagenerated by a first user and comprising annotation content and locationdata representing a location within the document for an annotation,wherein the annotation content is displayed in the second displayportion of the web page; the server updating the document by insertingan annotation indication of associated with the annotation in thedocument based on the location data wherein the annotation indication isdisplayed in the first display portion of the web page; and in responseto a request from a given user for the document, delivering the updateddocument with the annotation content and one or more instructionalelements to the given user for use in causing the given user's client toautomatically display the annotation content in the second displayportion when the corresponding annotation indication is displayed on thegiven user's client.
 49. The method of claim 48, further comprising:allowing different users to submit different annotations wherein thedifferent annotations are associated with related portions of thedocument; and combining the annotation indications in the first displayportion of the web page that are associated with the related portions ofthe document such that the combined annotation indications correspond tothe different annotations displayed in the second display portion of theweb page.
 50. The method of claim 48, wherein the document contains textarranged in one or more paragraphs, and wherein receiving location datacomprises receiving data identifying a given paragraph contained in thedocument.
 51. The method of claim 50, wherein receiving location datafurther comprises receiving data identifying an offset from thebeginning of the given paragraph.
 52. The method of claim 51, whereinreceiving location data further comprises receiving a selection lengthrepresenting a length of text selected by the first user for annotation.53. The method of claim 48, wherein receiving annotation data comprisesreceiving form data submitted by the first user in a markup language.54. The method of claim 48, wherein updating the document comprisesembedding the indication in the document.
 55. The method of claim 54,wherein embedding the indication comprise embedding the indication at alocation in the document identified by the location data.
 56. The methodof claim 54, wherein the document is structured in a markup language,and wherein embedding the indication comprises inserting indication datawithin a tag in the markup language.
 57. The method of claim 48,comprising: receiving the first server second annotation related to thedocument, the second annotation data generated by a user other than thefirst user and comprising second annotation content and second locationdata representing a second location within the document for a secondannotation. the server updating the document by inserting a secondindication of the second annotation in the document based on the secondlocation data; and in response to a request for the document from agiven user, delivering the updated document to the given user with theannotation content and second annotation content and a plurality ofinstructional elements for use in causing the given user's client todisplay the annotation content or second annotation content when theindication or second indication, respectively, is displayed on the givenuser's client.
 58. A method for displaying a web page having one or moreannotations, the method comprising: displaying a web page with at leasta first display portion and a second display portion; a user at a firstclient receiving from a server the web page having an item of annotationcontent created by a users at a client, a corresponding annotationindication embedded at a location within the web page, and one or moreinstruction elements related o the indication; a browser operating atthe first client displaying the web page in the first display portion ofthe display and allowing the user to navigate within the first displayportion so as to display different parts of the web page; the browserdisplaying the annotation indication in the first display portion whenthe location within the web page is displayed in the first displayportion of the display; and the browser using the one or moreinstructional elements to cause the annotation content to be displayedin a second display portion of the display when the annotationindication is displayed in the first display portion and to remove thedisplay of the annotation content when the annotation indication is notdisplayed in the first display portion.
 59. The method of claim 58,wherein the first and second display portions comprise a split screendisplay.
 60. The method of claim 58, wherein the browser allows the userto navigate tin the web page by jumping to the indication using a link.61. A method for automatically navigating a document in a display havingat least a first display portion and a second display portion, themethod comprising: displaying a web page with at least a first displayportion and a second display portion; receiving a plurality ofannotations related to the document, the annotations generated by aplurality of users at first client; associating each annotation with anannotation indication in the document; displaying in the first displayportion of the web page, the document and associated indications ofannotations; displaying in the second display portion of the web page,annotations corresponding to the indications: receiving, from a user ata second client, an input to navigate the plurality of annotations inthe second display portion, the input causing a first annotation of theplurality of annotations to be displayed in the second display portion,and in response to the input, automatically displaying, in the firstdisplay portion at the second client, the indication associated with thedisplayed first annotation.
 62. The method of claim 61, whereinassociating each annotation with an indication in the documentcomprises: identifying a portion of the document to which the annotationrelates; and associating the indication with the portion of the documentto which the annotation relates.
 63. The method of claim 62, whereinautomatically displaying the indication comprises displaying theindication with the portion of the document to which the annotationrelates.
 64. The method of claim 63, comprising adding each annotationto a data structure stored in memory, the data structured comprising alist of annotations relating to portions of one or more documents. 65.The method of claim 64, comprising storing the list of annotations on aserver remotely located from and communicatively coupled to the firstand second clients.
 66. The method of claim 65, wherein the list ofannotations comprises a list of bookmarks.
 67. The method of claim 66,wherein selecting an annotation from the list of bookmarks displays, inthe first display portion, at least a portion of a document to which theannotation is related and displays at least the selected annotation inthe second display portion.
 68. The method of claim 62, whereinassociating the indication comprises embedding the indication in theportion of the document to which the annotation relates.
 69. The methodof claim 68, wherein embedding the first indication comprises: receivinglocation data related to the portion of the document; processing thelocation data to determine a first location within the document relativeto a location of the portion within the document; and generating a newversion of the document, the new version of the document containing theindication embedded at the first location.
 70. The method of claim 69,wherein the location data comprises one or more from the groupcomprising: a document identifier, a section identifier, a chapteridentifier, a bookmark identifier, a portion length, and a portionoffset.
 71. The method of claim 69, further comprising replacing a firstversion of the document stored in memory with the new version of thedocument.
 72. The method of claim 71, wherein replacing a first versionof the document comprises overwriting a first version of the document.73. The method of claim 61, wherein receiving annotations comprisereceiving an annotation related to an image contained in the document.74. The method of claim 73, wherein receiving an annotation related toan image comprises receiving information identifying one or moresubjects of the image.
 75. The method of claim 74, the method furthercomprising associating the one or more subjects with the image.
 76. Themethod of claim 75, wherein associating the one or more subjects withthe image comprise updating a data structure stored in memory, the datastructure storing associations between one or more images and one ormore subjects of the one or more images.
 77. The method of claim 61,wherein the display comprises a browser window.
 78. The method of claim61, wherein the document comprise a web page.
 79. The method of claim61, wherein the document comprises a web page.
 80. The method of claim61, wherein the annotation comprises a text annotation.
 81. The methodof claim 80, wherein the text annotation comprises a comment related tothem document.
 82. The method of claim 61, wherein the annotationcomprises a graphical annotation.
 83. The method of claim 82, whereinthe graphical annotation comprises a photograph.